Information gain ratio is an information measure used in decision tree bulding algorithms, notably C4.5, that tries to remove some of the bias towards decsions with lareg numbers of alternatives. It is obtained by dividing the raw information gain (gain(C,G)) by the entropy of the decision grouping. That is
Information gain ratio = gain(C,G) / H(G)
where H(G) is the entropy of the division into the groups.
. This is similar to but different from normalised mutual information, as the latter divides by H(C)+H(G). In practice this means the information gain ratio it is weighted slightly more against larger numbers of groups compared with normalised mutual information.
Also known as normalised information gain
Links:
- Wikipedia: Information gain ratio
- link.springer.com: Induction of Decision Trees (Quinlan)